Skip to content

Add context quality scoring#221

Merged
ccf merged 2 commits intomainfrom
feat/context-quality-scoring
Apr 24, 2026
Merged

Add context quality scoring#221
ccf merged 2 commits intomainfrom
feat/context-quality-scoring

Conversation

@ccf
Copy link
Copy Markdown
Owner

@ccf ccf commented Apr 24, 2026

Summary

  • Add per-repository context quality scoring to maturity analytics, covering guide coverage/freshness, token efficiency, and sensor coverage
  • Surface context quality on the maturity Projects tab with top gap badges
  • Update roadmap and add backend/frontend tests for the new response shape and UI

Tests

  • ruff check .
  • ruff format --check .
  • bandit -r src/ -c pyproject.toml
  • pytest --import-mode=importlib -q (963 passed)
  • pytest --import-mode=importlib tests/test_maturity.py -q
  • frontend: ./node_modules/.bin/tsc --noEmit
  • frontend: ./node_modules/.bin/eslint src/pages/maturity.tsx src/components/maturity/context-quality-table.tsx src/components/maturity/__tests__/context-quality-table.test.tsx src/components/maturity/__tests__/maturity-summary.test.tsx src/types/api.ts
  • frontend: ./node_modules/.bin/vitest run --run src/components/maturity/__tests__/context-quality-table.test.tsx src/components/maturity/__tests__/maturity-summary.test.tsx

Note

Medium Risk
Adds new computed analytics and a new context_quality field to the maturity API response, which may affect dashboard consumers and has moderate risk of scoring/aggregation edge cases (null telemetry, repo metadata gaps).

Overview
Adds per-repository context quality scoring to maturity analytics, combining guide coverage/freshness, token/cache efficiency, and telemetry sensor coverage into a new context_quality dataset (with computed top-gap recommendations).

Surfaces this new dataset on the Maturity → Projects tab via a ContextQualityTable UI (empty state + gap badges), and extends the API schemas/types to include ContextQualityEntry in MaturityAnalyticsResponse.

Updates tests (backend maturity analytics + frontend table rendering) and marks the roadmap item as shipped.

Reviewed by Cursor Bugbot for commit 20bf93b. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Context quality sorted ascending instead of descending
    • Changed sort key from (score, -session_count) to (-score, -session_count) so highest-quality repos appear first, consistent with all other score-based sorts in the file.

Create PR

Or push these changes by commenting:

@cursor push 2b7c6f9946
Preview (2b7c6f9946)
diff --git a/src/primer/server/services/maturity_service.py b/src/primer/server/services/maturity_service.py
--- a/src/primer/server/services/maturity_service.py
+++ b/src/primer/server/services/maturity_service.py
@@ -1703,7 +1703,7 @@
                     )
                 )
     project_readiness.sort(key=lambda p: p.ai_readiness_score, reverse=True)
-    context_quality.sort(key=lambda row: (row.context_quality_score, -row.session_count))
+    context_quality.sort(key=lambda row: (-row.context_quality_score, -row.session_count))
 
     # Aggregate metrics
     avg_leverage = (

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit cd173e5. Configure here.

Comment thread src/primer/server/services/maturity_service.py Outdated
@ccf ccf merged commit 8456a6b into main Apr 24, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant